Lab-Based Response Assessment Algorithm Recapitulates Investigator's Response Assessment in the Phase 3 Bellini Trial

Background: Real-world data (RWD) derived from electronic health records are becoming increasingly important for deriving insights from clinical practice to complement findings from clinical trials. Response to treatment in multiple myeloma (MM) is assessed using the International Myeloma Working Group (IMWG) response criteria based on MM-specific laboratory measures (i.e. monoclonal [M] protein in serum and urine, free light chain [FLC] levels), as well as radiological images and bone marrow (BM) investigations when appropriate. As healthcare providers do not routinely collect all the information required by the IMWG response criteria, RWD data are often incomplete. Here, we present a derived real-world response (dR) algorithm based on IMWG criteria that accounts for lab measures routinely collected in the clinical care of MM patients and evaluate its ability to accurately assess response to treatment using clinical trial patient-level data.

Methods: Treatment response is assessed using a rule-based algorithm integrating longitudinal laboratory measures routinely captured in RWD (Xu et al. Pharmacoepidemiol Drug Saf 2021) as e.g. by the Flatiron MM database. The rules are based on 'relaxed' IMWG criteria, which entail exclusion of BM biopsies data and of imaging results, and reduction in either serum or urine M protein levels (rather than both) to assign partial response. This algorithm was applied to patient-level clinical trial data from the Bellini trial (Kumar et al. Lancet Oncol 2020) with response assignment made by an independent review committee (IRC) in the trial used as the 'ground truth'. Agreement between the IRC's and algorithm's assignments of response was estimated using Cohen's Kappa statistic.

Differences in overall response rate (ORR) between treatment and placebo arms were calculated using stratified Cochran-Mantel-Haenszel tests based on strata at randomization with number of prior lines of therapy and previous proteasome inhibitor treatment status variables used for stratification.

Results: The Bellini trial is a Phase 3 clinical trial with 2:1 design enrolling 194 and 97 patients in its treatment and placebo arms, respectively (291 patients in total). Regular assessments of M protein and FLC levels were performed (median 13 or 14 measurements), providing detailed trajectory of patient response (Table 1).

Comparison between the IRC's and algorithm's assignments of responders classified as Partial Response or better (PR+) resulted in almost perfect agreement with Cohen's Kappa 0.82 (274/291 assignments in agreement). It is worth noting that the Cohen's Kappa between the IRC's and Investigator's assessments is 0.85, indicating that the algorithm's error is within some expected uncertainty. Due to the exclusion of BM information, agreement decreased to 0.56 when depths of response were considered separately as opposed to grouping PR+ patients together with most cases being overestimated as complete response (CR) or stringent CR.

In assessing treatment effect, differences in ORR between the intervention and placebo arms in the trial based on IRC's assessment could be accurately recapitulated (68% [66/97] vs 71% [69/97] responders in the placebo arm and 82% [159/194] vs 87% [169/194] in the intervention arm for IRC's and algorithm's assessment, respectively). Based on these results, algorithm response assignment led to consistent conclusions about treatment efficacy in the Bellini trial. Implementation of criteria to characterize very good partial response or better (VGPR+) also led to a conclusion consistent with the Bellini trial IRC's assessment, while implementation of criteria of CR+ resulted in estimates of treatment efficacy that were higher, but directionally consonant (Table 2).

Conclusions: We present a fully automated rule-based algorithm for response assessment in MM relying on longitudinal lab measurements during treatment. The algorithm uses 'relaxed' IMWG criteria to account for routine clinical practice settings and demonstrates very high agreement with the assessment of trained clinicians, while reliably reproducing the efficacy analysis in the Bellini trial when ORR and VGPR+ are considered. We envision that, if the concordance observed here is confirmed in other independent cohorts, this algorithm can be used in the assessment of response in RWD and can facilitate response assessment when suboptimal amounts of data are available.

Figure 1

View largeDownload slide

Figure 1

View largeDownload slide

Close modal